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Abstract 

The paper reviews potential applications of functional Near-Infrared Spectroscopy (fNIRS), a 
well-known medical diagnostic technique, to monitoring the cognitive state of pilots with a focus on 
identifying ways to adopt this technique to airhome environments. We also discuss various fNIRS 
techniques and the direction of technology maturation of associated hardware in view of their potential 
for miniaturization, maximization of data collection capahilities, and user friendliness. 

Introduction 

The reduction of accident rates is of great importance in the face of increasing commercial aviation 
traffic. Even the most expert and conscientious pilots are susceptihle to making errors in stressful 
situations. Intelligent cockpits of the future will interact with operators in ways designed to reduce error 
prone states and mitigate dangerous situations at the edges of human performance. 

NASA’s Aviation Safety Program has recognized the role of pilots and their ability to respond in 
critical flight situations as one of the major elements in its continuing effort to improve the safety of 
commercial flight. It has also established, as a part of the Vehicle System Safety Technologies (VSST) 
Project, a theme of technical challenges to deal with crew decision making and responses in complex 
situations. The crew decision making (CDM) technical area includes a broad range of technologies 
dealing with safe operation of piloted airspace vehicles. It has been formulated in response to safety 
concerns coupled with the historical trends and the forecasted changes ushered in by advances in global 
air transportation. CDM is aimed at developing data and technologies that enable new capabilities for 
pilots to better understand and respond safely to complex situations, thereby increasing the pilots’ ability 
to avoid, detect, and recover from adverse events that could otherwise result in accidents/incidents. 

One important aspect of this is the development of reliable, sensitive and operationally-relevant 
metrics for the state of the “human in the loop” by noninvasive, portable, safe, and inexpensive means. 
The overall goal is to reduce the effects of performance decrement and improve safety by informing 
intelligent systems of the state of the operator. Appropriate injections of mitigations and operator support 
by an intelligent cockpit may then be made before cognitive decision-making abilities are compromised. 
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Monitoring of the aviation pilot’s situational awareness has become a subject of intensive studies over 
the past several years (Refs. 1 to 4). In the process of these studies, such diverse techniques as eye and 
gaze tracking and blood oxygen level monitoring have been considered. 

In this paper we will discuss issues associated with implementation in airborne environments of a 
spectroscopic blood oxygen level monitoring technique called functional Near-Infrared Spectroscopy 
(fNIRS). 


Functional Near-Infrared Spectroscopy (fNIRS) 

Near-Infrared Spectroscopy (NIS) is a medical tomographic technique to analyze structural properties 
of tissues (Refs. 5 and 6) and functional responses of a brain (Refs. 7 and 8) by non-invasively measuring 
concentration changes of oxy- and deoxy-hemoglobin (Hb02 and Hb) and cytochome oxidase (CtOx). 
Monitoring of those parameters also helps evaluating the cognitive state of the patient, for example in 
tracking the development of Alzheimer’s disease, and studying other brain-behavior relationships. The 
monitoring is usually done through utilization of two or more wavelengths of light selected such that the 
light propagation properties at those wavelengths are significantly different for oxygenated and 
deoxygenated tissues. In applications of NIS to detection of damaged or abnormal tissue regions, 
variations in the light absorption of tissue permit, for instance, an early detection of cancer. 

In functional Near-Infrared Spectroscopy (fNIRS), the light propagating at each wavelength through 
the tissue and blood-carrying vessels exhibits different properties, depending not only on the wavelengths 
used but also on the properties of the blood in the vessels. Processing information on the relative changes 
in concentration of Hb02 and Hb in blood vessels at particular locations in the brain indirectly determines 
the state of awareness of the subject or crewmember and his or her ability to make decisions or respond to 
situational changes. 

In general, the effectiveness of fNIRS applications to clinical analysis depends on several factors 
including design of electro-optic instrumentation, stability of human-machine mechanical contacts, 
“client” or subject cooperation, absorption and scattering of light by human tissue, availability of proper 
means to process and interpreted information, and other (Refs. 9 to 11). Adaptation of fNIRS to airborne 
environments adds other factors specific to the aircraft. For instance, while not considered of a significant 
importance in clinical environments, the size and weight of electro-optical instrumentation including 
signal processing will be limited when placing such instrumentation in the cockpit of an aircraft. There 
are three basic types of fNIRS instrumentation: continuous intensity, intensity modulated (or frequency 
domain), and time resolved. The first type, continuous intensity, is based on evaluation of the absorption 
of light passing through tissue. It provides intensity measurements at different wavelengths. The light 
sources in this case emit at constant intensity levels. In intensity modulated instrumentation, the light that 
enters the tissue is modulated and three parameters of the light passing through it are measured: intensity, 
phase, and modulation depth. Finally, the time resolved type employs a short pulse of light which after 
passing through tissue is spread in time. Each of these types has advantages and disadvantages with the 
continuous intensity being the most commonly used. A more detailed description of the fNIRS techniques 
and principles of their operations can be found elsewhere (Refs. 12 to 14). The hardware and hardware- 
related components of any fNIRS system can be loosely grouped as electro-optic instrumentation, 
headgear and fiber interfaces, and associated signal processing and software. In a particular system they 
are highly integrated and in many cases it is impossible to discuss one group of components without 
mentioning others. 

Table 1 outlines major areas of hardware and software related elements of fNIRS techniques. 
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TABLE 1.— MAJOR INSTRUMENTATION AND SOFTWARE RELATED ETP-MENTS OF fNIRS TECHNIQUES 


Elements 

Evaluation matrix 

Limitations 

Electro-optic Instrumentation 

Signal-to-noise ratio 
Small size and low weight 
Low power; Cost 

Operating wavelengths 
Sources and detectors 
Growing but limited vendor base 

Human-machine interface 
(headgear) and test subject 
cooperation 

User friendliness and safety 
Interface mechanical stability 

Acceptance by users/clients 
Insufficient usage data base 
Privacy concerns 

Software and signal processing 

Efficiency and credibility of processed 
information 

Data storage. Processor speed, Flight 
Automation firmware compatibility. 
Artifact removal algorithms 



Human Head Inside Human Head 

Figure 1 . — Schematic diagram of the fNIRS concept. 

Electro-Optic Instrumentation Element in fNIRS Systems 

The electro-optic instrumentation element is a key part of any fNIRS system. It consists of light 
sources emitting at particular wavelengths, photodetectors, and a system of optical fibers connecting the 
sources and detectors to a human head. Light from the sources reaches the head via optical fibers, gets 
scattered in the tissue, and a portion of the scatted light is coupled back into optical fibers and delivered to 
photodetectors for analysis. Studies have shown that in order to penetrate several centimeters through the 
tissue, the wavelength of the light should be between 600 and 900 nm (Refs. 15 and 16). Furthermore, 
absorption of light at the shorter range of the wavelengths is significantly affected by the presence of 
oxygen in the human tissue. At the same time, light at wavelengths between 800 and 850 nm experiences 
absorption that practically is not affected by the level of oxygenation (Refs. 17 and 18). 

The selection of sources and detectors is directly dependent on the light wavelength selected, and the 
configuration of the fNIRS system itself. 

Design of the instrumentation element of any fNIRS system is driven by the electrical power budget 
and associated with it the signal-to-noise ratio because of the selection of detector type. That is especially 
important in aerospace environments with limited weight and size as well as power availability. A 
schematic diagram representing the fNIRS concept is shown in Figure 1 . Light emitted from a light 
source at one of the operating wavelengths is coupled into one or several optical fibers and delivered to a 
light injector placed on a human head. Through the injector, the light is sent through the skull into the 
brain. In the brain, the light experiences significant scattering and some absorption, with the absorption 
especially being dependant on both the wavelength used and the levels of both deoxy- and oxygenated 
hemoglobin in the blood. Overall, just a small portion of light is being collected by the light collector 
placed also on the head centimeters from the injector. 
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In a classical continuous intensity fNIRS configuration, the laser sources emit continuously. 
However, in practice the sources are intensity or pulse-modulated at low frequencies to increase the 
signal-to-noise ratio and to provide multiplexing and demultiplexing (Refs. 19 and 20). While the 
continuous intensity configuration of fNIRS is not suited for absolute characteristics of the tissue optical 
properties (Ref. 21), it is the most common configuration used in hrain imaging (Ref. 22). 

In fNIRS systems employing high frequency intensity modulation, the modulation is achieved hy 
applying a sinusoidally varying current to a laser diode junction that is under a certain constant voltage 
bias (Refs. 23 to 25). Both the constant bias and sinusoidally varying current generate optical radiation 
with corresponding DC and AC components. In the intensity modulation fNIRS both parts of radiation 
can be measured and their dimensionless ratio used as an additional parameter for data evaluation 
(Ref. 26), Furthermore, an improved signal-to-noise ratio has been reported (Ref. 27). The modulation 
frequency used has varied from 100 to 800 MHz and the modulation depth decreased significantly with 
increase in the modulation frequency. If a primary focus of the fNIRS system design were to increase the 
modulation frequency an external modulation approach could be more appropriate. The external 
frequency modulation is usually provided by an electro-optic modulator incerted directly into the path of 
optical radiation in the fiber. The intensity of that radiation is changed by varying the current or voltage 
applied to the modulator. While such direct modulation increases the modulation depth at high 
frequencies, the capability of using the AC/DC ratio of intensities will be lost. 

fNIRS systems with pulse modulated light sources are capable of providing simultaneous spatial and 
temporal information about light propagation in human tissue (Refs. 28 and 29). Light sources used in 
those systems are usually pulsed laser diodes with pulse durations from 10 ps to 10 fs and repetition rates 
up to 100 MHz. The eomplexity of fNIRS systems with pulse modulated light sourees, is determined by 
their weight, real estate required to aeeommodate them, and finally the eost whieh makes them eurrently 
impraetieal for airborne applieations. 


Headgear 

Various designs of headgear for elinieal use have been reported (Refs. 30 to 32) with the main foeus 
on the funetionality of the entire system. With an inereasing interest in the development of an fNIRS 
system eapable of operating on a human in motion, the effieieney of the headgear design started to attraet 
attention (Ref. 33). 

The main sourees of power losses in the fNIRS system are eoupling losses on interfaees. Owing to the 
faet that the human tissue is a highly seattering medium, most of the losses oeeur during ingress and 
egress of the light and it propagation through the human head. Thus, the signal to noise ratio ean also be 
inereased by improving the eoupling of the injeetors to the sealp. This issue has been addressed in a 
number of publieations (Refs. 34 to 37). Design of headgear plays an important role in the aeeeptanee of 
fNIRS teehniques by the elients ineluding the pilot eommunity and safety regulators. Ease of operation 
and eomfort are also eriteria used in the design. 

To evaluate the use of gradient index (GRIN) lenses to improve signal injeetion and eolleetion, we 
eonsidered two eonfigurations. Gray matter phantoms were used to mimie the absorption and seattering 
properties of human brain tissue. Also, the light at two wavelengths, 690 and 830 nm, was brought to the 
injeetors via eommereially available multimode fiber mode fiber with 50 pm eore diameter. Furthermore, 
the GRIN lenses were used as eolleetors to gather light seattered in the phantom. 

Figure 2 shows the first two potential designs of optieal injeetors. In these two eases, as shown in the 
figure, the fibers are plaeed on the air-phantom interfaee and GRIN lenses are positioned in the plane 
normal to that interfaee. In Figure 2(a) the injeetor is a right angle prism (RA Prism) whieh eouples the light 
into the phantom by ehanging the direetion of the light propagation by 90°. Sueh eonfiguration permits 
eontinuing spreading of light after it exists the fiber and penetrates a wider but shallower area of the 
phantom. The injeetor shown in Figure 2(b) is a eombination of a right angle prism and GRIN lens. The 
ineorporation of a GRIN lens provides eollimation of the light and a deeper penetration within the phantom. 
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Figure 2. — Examples of optical injector configurations: (a) Injector with right angle prism; 
(b) Injector with right angle prism and GRIN lens. 


Optical Fiber 



Figure 3. — Example of the injector — collector configuration with GRIN 
lenses and mutual inclination. 

In the third configuration shown in Figure 3 the injector is a GRIN lens coupled to the fiber and the 
collector is also a GRIN lens and they both are facing the same phantom-air interface. A slight inclination 
of both injector and collector should enhance the directional coupling of light passing through the 
phantom material. However, this must be balanced against the subject’s comfort. 

While other configurations of injectors are potentially available, the GRIN lens approach has shown 
to be the most promising of those considered. Most designs and numerical analyses of the GRIN lens- 
based probes have been done for a single wavelength operation (Refs. 38 and 39). In fNIRS two 
wavelengths are being used and effects of GRIN lens on optical beams at those wavelengths are different. 
Figure 4 shows calculated spot diagrams of optical beams at two wavelengths, 690 nm (blue dots) and 
830 nm (red dots) after passing through a GRIN lens. The GRIN lens focuses the 690 nm wavelength 
beam at a distance of 7 mm from the exit aperture. The figure also shows clearly spreading of the beam at 
830 nm wavelength. 

Thus, owing to the fact that fNIRS employs two wavelengths, better GRIN lens designs are needed to 
improve the coupling efficiency of the light into the human tissue. Furthemore, tests need to be performed 
on human volunteers in order to assess this light injector design’s effectiveness at penetrating the 
cerebral-spinal fluid barrier and achieving cleaner scattered signals from the brain’s gray matter. 

Signal Processing 

Minuaturization of fNIRS instrumentation involves development of compact modulation schemes to 
drive two light sources in a timely prescribed manner. Figure 5 demonstrates a laboratory prototype of 
electronic setup that permits multiplexing of modulated optical signals generated by two lasers at 690 and 
830 nm. 
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Figure 4. — Calculated spot diagrams 
of optical beams at 690 nm (blue 
dots) and 830 nm (red dots) after 
passing through a GRIN lens. 
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Figure 5. — Laboratory electronic setup to multiplex two modulated optical signals 
generated by two lasers. 
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The fNIRS systems usually involve headgears with multiple light injectors and collectors. The 
injectors and collectors are connected correspondingly to light sources and detectors via optical fibers. In 
airborne environments, in order to provide optical power to multiple injectors from light sources and 
deliver optical power from multiple collectors to photodetectors efficient optical power delivery 
multiplexing and demultiplexing techniques should be devise. Such techniques, in addition to channeling 
optical signals between the source and detector, should also minimize the number of connectorizing 
fibers. Thus, a properly designed optical signal delivery and collection configuration permits construction 
of a compact and efficient system with the optimum number of sources and detectors and significaltly 
reduced number of connecting fibers. Figures 6 and 7 show examples of pulsed multiplexing techniques 
that deliver light at two operational wavelengths, 690 and 830 nm, to four optical injectors on one single 
headgear unit. 

Figure 6 has two sets of light sources, either light emitting diodes or lasers. Each set has one light 
source emitting light at 690 nm and the other one at 830 nm. Light sources in each set are pulse 
modulated in time with identical duty factor. In this figure the modulating frequencies for the light 
sources with similar wavelengths are shown to be identical. However, different modulating frequencies 
could be used as well to provide additional light source discrimination. The duty factors used in different 
sets may does not have to be identical. 



Figure 6. — Example of a combined frequency-pulse multiplexed optical radiation delivery system. 
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Figure 7. — Example of a combined frequency-pulse multiplexed optical radiation delivery system with asynchronous 
modulation of the sources. 


Figure 7 shows another configuration with multiplexed light sources. It differs from the one depicted 
in Figure 6 hy having additional asynchronous modulation of the light sources. 

Numerical Analysis 

For continuous wave fNIRS, the relative concentration change in oxy- and deoxy-haemoglohin on a 
single channel are related to the optical intensities at two operating wavelengths measured from the 
corresponding instrument (Ref. 40). Once relative changes in the concentration of the hemoglobin species 
have been determined, this data must be further post-processed to remove artifacts, filter noise, and 
ultimately classify the operator state. The use of fNIRS for state classification in an operational 
environment necessitates identification of data processing procedures which are demonstrably sufficient 
for delimiting between separable operator states, yet fast enough to operate under near real-time 
constraints. Solutions to the modified Beer-Lambert law can be computed by application of a single 
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matrix-vector product. However, the filtering and classification procedures used to further process fNIRS 
data may he collectively more computationally expensive, depending on the exact post-processing 
procedures selected. 

The ability to effectively distinguish between operator states using fNIRS is greatly enhanced by 
removing noise and artifacts from the data prior to conducting state classification. Many popular methods 
for filtering signals can be applied to fNIRS data to achieve this. Among them are wavelet filtering, blind 
source separation, and phase-adaptive Kalman filtering. 

Wavelet filtering methods have recently been used to great effect to aid in the processing of fNIRS 
data (Refs. 41 to 44). Among inherent features of discrete wavelet transforms that motivate their use for 
processing of fNIRS data are abilities to compress, denoise, and remove artifacts from single- and 
multidimensional digital signals. For example, the wavelet coefficients of digital signals tends to be 
sparse, allowing for efficient compression of data in wavelet transform domain. Wavelet transforms allow 
for the characterization of features within a signal to be examined at a variety of spatial or temporal 
scales, often referred to as Multi-Resolution Analysis (MRA). The wavelet coefficients of many digital 
signals tend to be uncorrelated in the transform domain, allowing for the de-correlation of independent 
signal features from background noise. And lastly, since each wavelet coefficient represents the content of 
the analyzed signal partially localized in both space (or time) and frequency, wavelet transforms provide 
simultaneous partial spatial and frequency localizations. 

Since applications of fNIRS typically involve the use of multiple channels of input data, denoising 
and artifact removal could also be potentially achieved using blind source separation techniques. Blind 
source separation techniques are algorithms which seek to systematically identify and separate unknown 
signals given a collection of mixed signal and noise data, without a priori knowledge of the underlying 
mixing process. These methods typically operate by iteratively minimizing some mutual measure of 
statistical independence (such as sample correlation) between lists of given data, under the assumption of 
an overall linear mixing process. 

Principal component analysis (PCA) and independent component analysis (ICA) are two popular 
methods of blind source separation and their applications to fNIRS for both offline and real time 
processing are well described in the literature (Refs. 45 to 49). 

Phase- adaptive Kalman filtering is another technique that has been recently introduced. It is specific 
to frequency domain fNIRS instrumentation (Ref. 50) and involves the application of an adaptive Kalman 
filter to the computed values of hemoglobin. In the application of the technique the intensity of the 
Kalman filter applied to each channel is tuned according to the variability of the corresponding phase shift 
channel. The principal behind this procedure is that high variation in the phase shift channel is associated 
with poor coupling of the associated optrode to the skin. Such a measure, taken over a short frame of 
time, provides a strong indicator of motion artifacts and an overall poor signal-to-noise ratio within that 
same block of time. The method has a number of advantages over alternative filtering operations that 
have been described. It does not introduce ringing or overshooting artifacts at the endpoints of the data, as 
certain combinations of wavelet filters and boundary conditions may introduce while conducting wavelet 
filtering (Ref. 51). In contrast to PCA or ICA, since this filtering is performed independently for each 
channel, it is applicable for cases where there are limited channels (or even a single channel) of interest. 

Discussion 

Currently used commercial fNIRS instrumentation is designed for hospitals and other clinical or 
laboratory environments. To make the fNIRS technology suitable for airborne applications, the 
conventional approaches should be reevaluated to make the instrumentation small, light, airworthy, and 
pilot-friendly. 

The design of the entire system should be based on the minimum resources availability. That includes 
space and weight limitations as well as low power consumption. While a broad range of optical sources, 
detectors, and fiber optic components is available for operational wavelengths in the communication 
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range, the wavelength range from 600 to 900 nm still lacks similar diversity. Thus, with a relatively 
limited vendor’ s base, configurations of the optical signal delivery and collection system should he 
designed to achieve the maximum signal-to-noise ratio and maximum efficiency. 

The design of headgear should provide, in addition to functionality, comfort and safety during pilot’s 
involvement in cockpit activities. The issue of comfort and client cooperation should he addressed hy 
minimizing the number of fiber optic links connecting the headgear with signal processing units including 
optical sources and detectors. Furthermore, the design of the headgear itself should be user friendly and 
not restrictive. Multiplexing schemes should be developed to minimize the number of fiber optic cables 
connecting the headgear with the signal processing equipment without negative effects on the signal-to- 
noise ratio. Moreover, development of appropriate wireless configurations should be considered to 
remove any constrains on the movement of pilots in the cockpit. 

Safety issues should also be considered in the process of selection of optical sources. While the 
current safety standards regarding minimum permissible exposure to optical radiation regulate the human 
skin exposure (Ref. 52), no such standards exist when the radiation is applied to human head. Effects of a 
long-term exposure of human scalp and brain tissue to high power highly focused optical radiation in 
600 to 900 nm wavelength range with potentially deep penetration have not been fully studied. 

The software for real time data processing should be developed under an assumption of constrained 
computational resources specific to aviation data systems. Numerical models and software for such 
conditions are being currently developed (Refs. 52 and 53) and their adaptation to cockpit applications of 
fNIRS would further enhance the technology. 

Finally, the cockpit environment requires a strict certification for any equipment brought to it. 
Therefore, the fNIRS instrumentation developed for airborne environments should have low 
electromagnetic emission and minimum electromagnetic interference with the already existing equipment. 

Conclusion 

While the fNIRS has found wide applications in the medical field the use of the technology on human 
subjects in airborne environments is practically nonexistent. One of the issues detrimental to the technology 
incertion into such environments is a lack of suitable instrumentation packages and associated software. 

The paper clearly identifies major areas in the development and implementation of instrumentation 
and software related elements for airborne fNIRS systems. It also suggests a path for the technology 
maturation as articulated in the discussion section. As fNIRS is shown to be useful for detecting cognitive 
state and improving aviation safety, the technical maturation of these elements specifically toward use in 
airborne environments will become more warranted. 
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